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DETAILED ACTION 
Specification 

The disclosure is objected to because of the following informalities: 

• Equations 3, 4, and 5 should be reformatted. 
Appropriate correction is required. 

Claim Objections 
Claim 1 is objected to because of the following informalities: 

• On line 21 of claim 1 , "the memory" should be replaced with "a memory" since no 
instance of memory has been previously recited in the claim. 

• On line 29 of claim 1 , "N" is not previously defined. However, it is understood to 
be the predetermined number referred to in step (f). If this is indeed the case, 
then "(N)" should follow "predetermined number" found in line 26. 

Appropriate correction is required. 

Claims 7 and 42 are objected to because of the following informalities: 

• On line 2 of both claims 7 and 42, "the selected corner points" should be 
replaced with "selected corner points" since "corner points" are not recited in 
claim 1 or 36, respectively. 

Appropriate correction is required. 
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Claim 47 is objected to because of the following informalities: 

• Claim 47 as written is incorrect because it is dependent upon itself, 
that claim 47 should depend on claim 46. 

Appropriate correction is required. 

Claim 54 is objected to because of the following informalities: 

• Claim 54 as written is incorrect because it is dependent on claim 55, which has 
not been previously recited. It is believed that claim 54 should instead depend 
on claim 53. 

Appropriate correction is required. 

Claims 12-14, 26, and 46 are objected to because of the following informalities: 

• The equations in each of these claims do not have an equal signs. 

• In addition the equation of claim 26 should be reformatted. 
Appropriate correction is required. 

A series of singular dependent claims is permissible in which a dependent claim 
refers to a preceding claim which, in turn, refers to another preceding claim. 

A claim which depends from a dependent claim should not be separated by any 
claim which does not also depend from said dependent claim. It should be kept in mind 
that a dependent claim may refer to any preceding independent claim. In general. 



Page 3 



It is believed 



Application/Control Number: 10/063,834 Page 4 

Art Unit: 2623 

applicant's sequence will not be changed. See MPEP § 608.01 (n). Claims 13, 14, 18, 
and 49 are separated by claims without the same dependency. 



Claim Rejections - 35 USC §112 

The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

Claim 2 is rejected under 35 U.S.C. 112, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. 

Claim 2 recites the limitation "calculating correspondence" in line 2. There is 
insufficient antecedent basis for this limitation in the claim. It is believed that calculating 
correspondence refers to the step of calculating quality of alignment as recited in claim 
1. 

Claims 12-16, 36, 46, and 47 are rejected under 35 U.S.C. 112, second 
paragraph, as being indefinite for failing to particularly point out and distinctly claim the 
subject matter which applicant regards as the invention. The variables L2, L3, L4, L5, 
M, Q, and L are not clearly defined. 



Ciaim Rejections - 35 USC § 103 
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The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1, 4-27, 30, 31, 36, 39-53, and 55-58 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over the combination of Szeliski et al. (PN 5,611,000) and Roche 
et al. (PN 6,539,127). (hereinafter Szeliski and Roche respectively) 

Regarding Claim 1: Szeliski et al. (5,61 1,000) discloses a method for registering a 
plurality of image sets, wherein each of the plurality of image sets comprises at least 
one image, and wherein each of the plurality of image sets contain spatially overlapping 
areas of an imaged subject with at least one of the remaining plurality of the image sets, 
the method comprising: 

• selecting a reference image set and an evaluation image set from said plurality of 
image sets, wherein the evaluation image set is to be aligned with the reference 
image set [Figure 14. Images 1 (100) and 2 (101) are the reference and 
evaluation images respectively (col. 10 lines 22-23).]; 

• selecting a methodology for comparing of the registration between the reference 
image set and the evaluation image set [Figure 14. The image comparator (920) 
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is selected for comparing the reference image (100) and the evaluation image 
(101) (col. 10 line 19).]; 

• selecting one or more point locations on the evaluation image set for tracking 
image movement [Figure 4. Control vertices (201-263) belonging to the 
evaluation image (101) are used for tracking image movement (col. 23-26).]; 

• selecting one or more fixed reference points for comparison with the one or more 
point locations on the evaluation image set [Figures 5 and 6. A pixel (130) at a 
fixed or predetermined position in the first image (100) is selected so that it may 
be used in finding the corresponding pixel in the second image (101) (col. 7 line 
61 -col. 8 line 1).]; 

• selecting type of transformation to apply to the evaluation image set for aligning 
the evaluation image set with the reference image set [Affine transformation 
could be used to displace pixels of the second image (1 01 ) (col. 5 lines 23-25).]; 

• a) calculating quality of alignment between the reference image set and the 
evaluation image set using a selected quality of alignment methodology [The cost 
function (Equation 6, col. 5 lines 57-67) calculates the quality of alignment (col. 7 
lines 52-53).]; 

• b) calculating a location value (C) from one or more points on the evaluation 
image set with respect to the selected one or more fixed reference points and 
storing the calculation in the memory [Equation 6. The result of the cost function 
(E), which represents the location value (C), is a value relating a selected pixel 
on the derived image to a corresponding fixed reference pixel on the actual 
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image (col. 5 lines 49-56). The memory (42) is used to store the steps and 
therefore stores the new locations (col. 4 lines 16-18).]; 

• c) calculating a next transformation to apply to the evaluation image [Figure 14. 
The error integrator (940) calculates the vertex gradients (904) to be used for 
transforming the second image (101) (col. 10 lines 48-52).]; 

• d) applying the transformation to at least a subset of the evaluation image set 
[Figures 4 and 14. The vertex displacer (950) uses the vertex gradients (904) to 
displace the control vertices (201-263) thereby applying the transformation to the 
second image (101 ) (col. 10 lines 53-57).]; 

• e) calculating a convergence value (V) for the current iteration (i), and storing the 
convergence value to the memory [The convergence value is the calculated rate 
of change, or gradient, of the cost (col. 9 lines 37-42).]; 

• f) performing steps (a), (b) (c), (d) and (e) until at least a predetermined number 
of correspondence calculation iterations have been performed; and 

• g) repeating steps (a), (b), (c), (d), and (e) if a total number of iterations (i) 
performed < N and the convergence value (V) < (t), wherein (t) is a threshold 
value. 

Szeliski does not disclose step f. 

However, Roche discloses a method of automatic image registration wherein image 
transformation is repeated at least a predetermined number of times (col. 9 lines 3-5). 
Szeliski does not disclose step g either. 
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However, Roche discloses a method of automatic image registration wherein image 
transformation is repeated as long as the convergence value remains below a 
threshold, referred to as a convergence criterion (col. 7 lines 57-67). 
Szeliski teaches determining the final transformation iteration by comparing the cost 
with a desired minimum (col. 10 lines 5-6). It would be obvious to one of ordinary skill in 
the art to modify this teaching to use the convergence value instead of cost as taught by 
Roche, especially since Szeliski already computes the convergence value (see step d). 
One would be motivated to make this modification to avoid making excessive 
transformations. In addition, it would be obvious to one of ordinary skill in the art to use 
a predetermined number of iterations as another determination for ending the 
transformation iterations. One would be motivated to use a large predetermined 
number in addition to the determination on convergence value to end the iterations in 
case a convergence could not be reached in a timely manner. 
The analogous arguments of claim 1 are applicable to claims 36 and 55-58. 

Regarding Claim 4: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 2, wherein the image sets to be registered are capable of 
differing in at least one of temporally, modality, acquisition orientation, and 
dimensionality [The reference image (100) and evaluation image (101) differ temporally 
(col. 4 lines 4-6).]. 

The analogous arguments of claim 4 are applicable to claim 39. 
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Regarding Claim 5: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 1 , wherein the image sets to be registered are capable of 
differing in at least one of temporally, modality, acquisition orientation, and 
dimensionality [The reference image (100) and evaluation image (101) differ temporally 
(col. 4 lines 4-6).]. 

The analogous arguments of claim 5 are applicable to claim 40. 

Regarding Claim 6: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 1 , wherein the selecting of point locations includes selecting 
corner points of the evaluation image set as the selected location points [Figure 8. The 
control vertices (216, 217, 223, 224) are in the corners of a patch (314') in the 
evaluation image (101) (col. 8 lines 33-35). It is possible that the patch (314) could be 
the entire evaluation image (101 ). The positions of the control vertices are initially 
selected for feature tracking (col. 12 lines 35-45) and may be corners (col. 12 line 40).]. 
The analogous arguments of claim 6 are applicable to claim 41. 

Regarding Claim 7: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 1, further including using the selected corner points of the 
evaluation image set before the initial transformation as the fixed reference locations 
[Figure 8 shows that the corner points used as the control vertices (216, 217, 223, 224) 
on the evaluation patch (314'), correspond to the corner points, which are selected as 
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the fixed points, of the reference patch (314) belonging to the reference image (100) 
(col. 12 lines 35-45).]. 

The analogous arguments of claim 7 are applicable to claim 42. 

Regarding Claim 8: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 1, wherein the location value (C) is an average distance 
between the one or more selected point(s) at the current iteration and the one or more 
fixed reference locations [Equation 6. The result of the cost function (E), which 
represents the location value (C), gives the average distance between a selected pixel 
on the derived image and a fixed reference pixel on the actual image at a current 
iteration (col. 5 lines 49-52).]. 

The analogous arguments of claim 8 are applicable to claim 43. - 

Regarding Claim 9: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 1 , wherein the convergence value (V) is calculated from the 
location values (C) [Equations 19 and 20. The convergence value (V) is the rate of 
change of E, which represents the location value (C) (col. 9 lines 36-42).]. 
The analogous arguments of claim 9 are applicable to claim 44. 

Regarding Claim 10: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 9, wherein location value (C) is low pass filtered to remove 



Application/Control Number: 10/063,834 Page 11 

Art Unit: 2623 

noise [Low pass filtering is done to the image (100) as a pre-processing step thereby 
removing noise from the location value (C) (col. 4 lines 41-46).]. 

Regarding Claim 11: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 1 , wherein location value (C) is low pass filtered to remove 
noise [Low pass filtering is done to the image (100) as a pre-processing step thereby 
removing noise from the location value (C) (col. 4 lines 41-46).]. 
The analogous arguments of claim 1 1 are applicable to claim 45. 

Registering Claim 12: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 1 , wherein the convergence value (V) for each iteration (i) is 
defined as shown by the equation given in claim 12 [The horizontal and vertical flow 
gradients represent the convergence value and are defined by equations 19 and 20, 
which correspond to claim 12's equation (col. 9 lines 41-50).] 
The analogous arguments of claim 12 are applicable to claim 46. 

Regarding Claim 13: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 3, wherein the convergence value (V) for each iteration (i) is 
defined as shown by the equation given in claim 13 [The horizontal and vertical flow 
gradients represent the convergence value and are defined by equations 19 and 20, 
which correspond to claim 13's equation (col. 9 lines 41-50).] 
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Regarding Claim 14: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 8, wherein the convergence value (V) for each iteration (i) is 
defined as shown by the equation given in claim 14 [The horizontal and vertical flow 
gradients represent the convergence value and are defined by equations 1 9 and 20, 
which correspond to claim 14's equation (col. 9 lines 41-50).] 

Regarding Claim 15: Roche discloses the method for registering a plurality of images 
sets according to Claim 14, wherein convergence is obtained when absolute value of 
(V) is less than or equal to t for at least L3 of the last L4 iterations such that L3 is less 
than or equal to L4 [The test module (9) terminates the iterations of the method when a 
maximum predetermined number of iterations are made (col. 9 lines 50-55). It would be 
obvious to modify the test module (9) to also set a minimum number of iterations to be 
processed. One would be motivated to make this modification to improve the quality of 
alignment when more computational processing and time are affordable.]. 
The analogous arguments of claim 1 5 are applicable to claim 47. 

Regarding Claim 16: Roche discloses the method for registering a plurality of images 
sets according to Claim 1 , wherein convergence is obtained when absolute value of (V) 
is less than or equal to t for at least L3 of the last L4 iterations such that L3 is less than 
or equal to L4 [The test module (9) terminates the iterations of the method when a 
maximum predetermined number of iterations are made (col. 9 lines 50-55). It would be 
obvious to modify the test module (9) to also set a minimum number of iterations to be 
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processed. One would be motivated to make this modification to improve the quality of 
alignment v\/hen more computational processing and time are affordable.]. 

Regarding Claim 17: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 1 , wherein the transformation of the evaluation image set is one 
of affine, rigid, deformable, and perspective [Affine transformation is used (col. 5 lines 
23-25).]. 

The analogous arguments of claim 17 are applicable to claim 48. 

Regarding Claim 18: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 4, wherein the transformation of the evaluation image set is one 
of affine, rigid, deformable and perspective [Affine transformation is used (col. 5 lines 
23-25).]. 

The analogous arguments of claim 18 are applicable to claim 49. 

Regarding Claim 19: Roche discloses the method for registering a plurality of image 
sets according to Claim 1 , wherein the transformation applied to the evaluation image 
set is determined by an optimization method [Figure 1 . The calculation module (8) 
determines the registration transformations utilizing an optimization algorithm (col. 8 
lines 45-49).]. It would be obvious to one of ordinary skill in the art to modify Szeliski's 
vertex adjuster with an optimization algorithm because the cost function (equation 6) 
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can have many local optimal solutions (col. 6 lines 1-5) and one would be motivated to 
choose the most efficient solution. 

Regarding Claim 20: Roche discloses the method for registering a plurality of image 
sets according to Claim 19, wherein calculating correspondence further includes at least 
one of a Powell's method optimizer, a steepest gradient descent optimizer, a LBFGS 
optimizer, a Levenberg-Marquardt optimizer, a conjugate gradient optimizer, and a 
quasi-Newton optimizer [The optimizer algorithm could be the Powell type (col. 8 lines 
48-57).]. 

i 

Regarding Claim 21 : Szeliski discloses the method for registering a plurality of image 
sets according to Claim 1 , wherein an image set comprises one or more images [Figure 
14. A first image (100) and second image (101) are used for registration (col. 10 lines 
22-23).]. 

The analogous arguments of claim 21 are applicable to claim 50. 

Regarding Claim 22: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 21, wherein the image set comprises data relating to functional 
measurements [The image set may include images from medical scanners (col. 3 line 
67 - col. 4 line 3). It is well known that images from medical scanners offer data with 
functional measurements used for many different diagnostic procedures.]. 
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Regarding Claim 23: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 21 , wherein the image set comprises data relating to anatomical 
data [The image set may include images from medical scanners, which inherently 
contain anatomical data (col. 3 line 67 - col.. 4 line 3).]. 

Regarding Claim 24: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 1 , wherein the image set comprises data relating to functional 
measurements [The image set may include images from medical scanners (col. 3 line 
67 - col. 4 line 3). It is well known that images from medical scanners offer data with 
functional measurements used for many different diagnostic procedures.]. 
The analogous arguments of claim 24 are applicable to claim 51 . 

Regarding Claim 25: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 1, wherein the image set comprises data relating to anatomical 
data [The image set may include images from medical scanners, which inherently 
contain anatomical data (col. 3 line 67 - col. 4 line 3).]. 
The analogous arguments of claim 25 are applicable to claim 52. 

Regarding Claim 26: Szeliski discloses the method for registering a plurality of image 
sets according to Claim 1 1 , wherein the low-passed filter is defined as given by the 
equation in claim 26, wherein LI is the length of the low-pass filter [The location value 
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(C) is inherently altered according to the equation as shown in claim 26 when boxcar 
filtering is utilized to reduce the impact of noise (col. 4 lines 44-46).]. 

Regarding Claim 27: Roche discloses the method for registering a plurality of image 
sets according to Claim 1 , wherein the calculation of the convergence threshold (t) is 
dynamically determined based on the data [The convergence threshold is part of the 
optimization algorithm and therefore would be dynamically determined based on the 
data to optimize the registration (col. 7 lines 64-66).]. 
The analogous arguments of claim 27 are applicable to claim 53. 

Regarding Claim 30: Roche discloses the method for registering a plurality of image 
sets according to Claim 1 , wherein the quality of alignment is voxel based [Accuracy is 
calculated with respect to voxel size (col. 5 lines 10-12).]. It would be obvious to one of 
ordinary skill in the art to modify Szeliski to create a spline of voxels and therefore 
determining quality of alignment based on corresponding voxels (col. 5 lines 58-62). 
Furthermore, one would be motivated to make this modification to improve alignment 
accuracy. 

Regarding Claim 31 : Roche discloses the method for registering a plurality of image 
sets according to Claim 30, wherein the voxel based quality of alignment is further 
based on intensity of pixels of the image [The correlation ratio, which is used to 
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determine the quality of alignment, is based on random variables representing intensity 
components of the images being aligned. 

Claims 2, 3, 32, 33, 37, and 38 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Szeliski and Roche in view of Viola et al. ("Alignment by 
Maximization of Mutual Information"), (hereinafter Viola) 

Regarding Claim 2: Szeliski and Roche disclose the method for registering a plurality of 
image sets according to Claim 1 , but do not disclose the step of calculating a quality of 
alignment including mutual information. 

However, Viola discloses an alignment method wherein calculating correspondence 
further includes calculating Mutual Information between the reference image set and the 
evaluation image set [A maximum of the mutual information of a model u (evaluation 
image) and image v (reference image) is calculated to estimate the transformation (p. 
17, Section 2.1, lines 1-3).]. It would be obvious to one of ordinary skill in the art to 
modify the calculating of alignment quality with mutual information since Szeliski 
discloses the desire to perform image registration without a priori knowledge (col. 2 
lines 52-55). Furthermore one would be motivated to make this modification to align 
medical images when one image Is an MRI and the other is a CT image. 
The analogous arguments of claim 2 are applicable to claim 37. 
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Regarding Claim 3: Viola discloses the method for registering a plurality of images sets 
according to Claim 2, wherein the Mutual Information calculation further includes 
Stochastic sampling to compute the Mutual Information [The maximum of mutual 
information uses a stochastic sampling (p. 18, Section 2.3.1, lines 1-2).]. 
The analogous arguments of claim 3 are applicable to claim 38. 

Regarding Claim 32: Viola discloses the method for registering a plurality of image sets 
according to Claim 31 , wherein Mutual Information is utilized to determine quality of 
alignment [The mutual information is maximized to determine the transformation that 
yields the best quality of alignment (p. 17, Section 2.1 , lines 1-3).]. It would be obvious 
to one of ordinary skill in the art to modify Roche's correlation ratio to utilize mutual 
information as is well known in the art of image registration. 

Regarding Claim 33: Viola discloses the method for registering a plurality of image sets 
according to Claim 32, wherein Mutual Information further includes stochastic 
approximation [The maximum of mutual information uses a stochastic sampling (p. 18, 
Section 2.3.1, lines 1-2).]. 

Claims 34 and 35 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Szeliski and Roche in view of Moshfeghi (PN 5,633,951). 
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Regarding Claim 34: Szeliski and Roche disclose the method for registering a plurality 
of image sets according to Claim 1 , but do not disclose the use of features to determine 
the quality of alignment. However, Moshfeghi discloses an image registration method, 
wherein the quality of alignment is feature based [The contours, representing features, 
of the images to be registered are extracted creating surfaces that are used for 
alignment (col. 2 lines 45-51 ).]. It would be obvious to one of ordinary skill in the art to 
modify Szeliski's quality of alignment determination to use features of the image to 
further improve the accuracy of image registration. 

Regarding Claim 35: Moshfeghi discloses the method for registering a plurality of 
image sets according to Claim 34, wherein the feature based quality of alignment is 
further based on edges extracted from the image [The contours, or edges, of the 
images to be registered are extracted creating surfaces that are used for alignment (col. 
2 lines 45-51).]. 

Allowable Subject Matter 

Claims 28, 29, and 54 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 



Conclusion 
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The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

• Miller et al. (PN 6,009,212) is cited for teaching image registration by selecting 
points and iteratively transforming an image until a convergence threshold or 
number of iterations is reached. 

• Suri (PN 6,71 8,055) is cited for teaching image registration by repeating 
transformations and using mutual information. 

• Defaux et al. (US 2002/01 86425) is cited for teaching image registration using 
multiple sweeps of images to be aligned. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Craig W. Kronenthal whose telephone number is (571 ) 
272-7422. The examiner can normally be reached on 8:00 am - 5:00 pm / Mon. - Fri.. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on (571 ) 272-7414. The fax phone number for 
the organization where this application or proceeding Is assigned is 703-872-9306, 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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